In the cross-referenced related U.S. application (Ser. No. 07/826,754), a system and method are disclosed for the dynamic substitution of a chosen image or indicia into a preselected portion of an existing video image, such as may be depicted in a television program. That system, which is characterized as an "Electronic Billboard", operates to detect part or all of an object or objects within a video image and to thereafter use the position of that object or part thereof as a reference. The Electronic Billboard system then operates to replace a portion of the original video image, as identified by the position reference, with another still or video image chosen by the operator. The system further operates to position that replacement image in exact correspondence with the replaced portion of the original video image, such that the final composite image appears to a viewer as though it were the actual image being recorded by the video camera which created the original video image. In other words, the replacement image is seamlessly and realistically incorporated into the video image of the original event. Moreover, because of the dynamic substitution capability of the Electronic Billboard system, the appearance of the replacement image will continuously conform to the appearance of the original scene when that scene is moved, panned, magnified, zoomed, or otherwise altered in size or perspective.
As also explained in the disclosure for the "Electronic Billboard" invention, the pattern recognition software for the invention utilizes the "Burt Pyramid" algorithm, which, as is well known to those skilled in the art, is a very powerful methodology for processing, analyzing, and/or synthesizing video signals. Nonetheless, even though such pattern recognition software thereby achieves a processing speed which often permits real-time processing for the image replacement methodology of the invention, it was noted that, in some circumstances, a short time delay may be necessary in the transmission of the composite video image to accommodate the necessary processing. In such a circumstance, a frame-store means would be used with the Electronic Billboard in order to temporarily store a small number of video frames, thereby incorporating a time delay mechanism into the system.
In a preferred embodiment of the Electronic Billboard invention, the video image to be processed is a televised sports event. In such an application, the identified portion of the original video image to be replaced may be the image of the playing surface, or a portion thereof, and/or stationary surfaces which are part of a structure, such as a stadium, proximate to that playing surface. With this embodiment, the replacement image would typically be an advertising message. A useful example of such a preferred embodiment is a televised tennis match. There the actual video image of one or both halves of the tennis court, on which the match is being played, might be replaced by use of the Electronic Billboard, such that a viewer of the televised tennis match would see an advertisement, as on a billboard, appearing on the tennis court. It will, of course, be possible to locate the replacement image within the bounds of certain marked areas of the court, so as not to interfere with the television viewer's ability to determine when a tennis ball lands within or without such a marked area, or alternatively, to eliminate the substitute image during periods of active play on the court in question.
Where a televised program is distributed to a number of remote geographic locations, as from a network program origination location to a variety of network affiliates, it will, of course, be possible to locate such an Electronic Billboard system at the originating site (or the site from which the original program is electronically distributed), as well as at any position downstream from that originating location in the chain of distribution of that program to an user. Alternatively, it is possible to separate the Electronic Billboard functions of object detection/recognition and of image insertion/replacement, and provide two distinct systems, a "master" and a "slave" system. The "master" system, which does the initial image detection and recognition, and is essentially the front-end of the unitary Electronic Billboard system, is situated at the broadcast originating location. The second system, the "slave", which will be situated at a downstream location, is essentially the back-end of the unitary Electronic Billboard system and operates by receiving and operating on various parameters sent from the master system--in particular, information pertaining to the precise location of the inserted image, such as the coordinates of the origin of the site at which to locate the image to be inserted, and any other data necessary for the slave to successfully carry out the insertion of the required replacement image in the proper place and manner. The parameters passed from master to slave are essentially the same parameters transferred internally in the unitary Electronic Billboard. These parameters constitute a small volume of data compared to television signals. As such, in the master/slave arrangement, these parameters could either be transmitted from master to slave via a separate telephone link up, or they could be incorporated in the existing video signal, such as in the vertical blanking interval, as is done in the well known teletext broadcast systems, exemplified by the British Broadcasting Corporation's Ceefax service.
The program originator of a program such as described above will generally have a fight to control the distribution of the program at downstream locations, particularly with respect to advertising associated with such a program. Where such advertising is implemented through an Electronic Billboard system by a replacement of portions of the video image of the actual program content, as opposed to breaks in the program content, the exercise of such a control right becomes a critical concern.
In general, the downstream operator will distribute such a program pursuant to a license from the program originator and subject to financial obligations associated with such a license. Because the sub-distribution rights, and particularly the advertising rights, are likely to vary between different downstream operators, it is necessary that a control mechanism be available to the program originator for assuring compliance by each downstream operator with the terms of the particular license arrangement existing between the program originator and that downstream operator. However, with the unique capabilities of the Electronic Billboard system, there are presently no control mechanisms known for adequately protecting a program originator's rights in the use of such a system. It is, therefore, a principal object of this invention to implement such a control mechanism for use with the Electronic Billboard system.
It is expected that the television industry will adopt practices for the use of the electronic billboard broadly similar to those already established for advertising inserted in breaks between programs or program segments. Typically, the existing practice consists of a rights holder to a particular program or event transmitting that program or event to other broadcasters (downstream broadcasters) for use in their markets. The rights holder may or may not include advertising which the downstream broadcaster is required to broadcast. The downstream broadcaster may or may not have negotiated the right to use some or all of the time slots in and between program segments to insert advertising of its own, depending on the terms of an agreement (or license) between the network or original broadcaster and the downstream broadcaster for the use of the program.
As is well known, the television industry is highly dependent on the legitimate sharing of programs, whether within a network structure or not. Moreover, a critical aspect of these "sharing" relationships is that the actual sharing of programs takes place as intended (and as defined by contracts between the affected parties). Accordingly, much effort is put into monitoring what is actually transmitted by all of the broadcasters. With conventional broadcasting methods, accounting firms are retained for the task of monitoring the broadcasting of the programs, including the advertising content, the location of the advertising and the nature of competing advertising, in order to determine what programs and advertisements are actually transmitted.
It would of course be possible to apply these conventional control and audit techniques to program sharing and advertising implemented through an Electronic Billboard system, but the nature of the image substitution functionality for an Electronic Billboard system imposes two serious obstacles to the use of such conventional methods. First, the image substitutions take place throughout the entire program and not just at specific breaks, requiring that a great deal more broadcast or "air time" be monitored, with a likely substantial increase in the cost of such monitoring. Second, the subtle and seamless nature of the Electronic Billboard image substitutions can be expected to make the monitoring process far more difficult in practice, with consequent loss of accuracy--possibly a significant reduction in accuracy.
To overcome these limitations in an application of conventional methodologies to the control and monitoring of downstream program/advertising content using an Electronic Billboard, a novel means for authorizing, controlling, and monitoring the insertion of advertising indicia and other video images into the broadcast stream by an Electronic Billboard system has been developed and is disclosed herein. The disclosed control means may also be applied for maintaining the integrity of either or both the products being advertised and the event being broadcast. Additionally, this control means will be selectably usable, in conjunction with an Electronic Billboard system, to create a vertical stratification of the advertising market--such as the allowance of only certain modes of operation for the Electronic Billboard system, as, for example, the use of the substitution functionality only at times when active play is not taking place or only when certain players are or are not included in the televised scene.